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PICTURE ENCODING METHOD, PICTURE 
ENCODING APPARATUS AND PICTURE 
RECORDING MEDIUM 

This application is a continuation of application Ser. No. 5 ' 
08/351,262, filed as PCT/JP94/00610 Apr. 11, 1994 pub- 
lished as W094/24822 Oct 27, 1994, now abandoned. 

TECHNICAL FIELD 

This invention relates to a picture encoding method, a 10 
picture encoding apparatus and a picture recording medium, 
and more particularly to a picture encoding method, a 
picture encoding apparatus and a picture recording medium 
which are used in a system for implementing encoding for 
storage to a video signal of moving picture to record the 15 
coded signal onto a picture recording medium such as an 
optical disc, a magnetic disc, or a magnetic tape, etc, or a 
system for transmitting a video signal of moving picture 
through a transmission path, 

20 

BACKGROUND ART 

Hitherto, in a system for transmitting a video signal of 
moving picture to remote place, for example, as-in-the- 
television conference system, the television telephone 
system, etc., or a system for recording a video signal of 25 
moving picture onto a picture recording medium such as an 
optical disc, a magnetic disc or a magnetic tape, etc., or 
reproducing a recorded video signal of moving picture, there 
is adopted for the purpose of efficiently utilizing transmis- 
sion path (or picture recording medium), a scheme to 
implement so called efficient encoding to a video signal by 
making use of correlation between lines or correlation 
between frames that video signal has to reduce redundancies 
in the spatial axis direction and the time axis direction to 
transmit only significant information, thus to improve trans- 35 
mission efficiency. 

For example, in encoding processing in the spatial axis 
direction (hereinafter intra-frame coding processing), e.g., 
correlation between lines of a video signal is utilized as ^ 
shown in FIG. 7A. In the case of attempting to transmit 
respective pictures PCI, PC2, PC3 . . . constituting amoving 

picture at times tl, t2, t3 picture data to be transmissionV 

processed is caused to undergo one-dimensional coding, 
eg., within the same scanning line, or a picture is divided 45 
into, e.g., a plurality of blocks to allow picture data of 
respective blocks to undergo two-dimensional coding to 
thereby carry out data compression, thus to improve trans- 
mission efficiency. 

Moreover, in coding processing in the time axis direction 
(hereinafter referred to as inter-frame coding processing), 
inter-frame correlation of video signal is utilized to 
determine, by so called predictive coding, for example, 
picture data PC12, PC23 . . , comprised of deficiencies (so 
called predictive errors) of picture data every corresponding 55 
pixels between adjacent pictures PCI and PC2, PC2 and 
PC3 ... in succession to transmit these picture data PC12, 
PC23, ... to thereby carry out data compression, thus to 
improve transmission efficiency. 

Thus, as compared to the case where all picture data of 60 
pictures PCI, PC2, PC3 ... are transmitted, a video signal 
can be transmitted by extremely lesser data quantity. 

Further, in the predictive coding in the above-described 
inter-frame coding processing, motion compensated predic- 
tion Is used, e.g.. in macro block units in order to further 65 
improve efficiency. Namely, e.g., in the case where a person f 
at the central portion of picture moves, or the like, motion > 



(movement) of an object moving in the picture is detected to 
correct position of picture data used for prediction in the 
former picture by that motion to carry out predictive coding, 
thereby making it possible to improve coding efficiency. 

5 However, even when such motion compensated prediction is 
employed, many data must be transmitted with respect to the 
portion where an object moves and appears from behind. In 
view of this, not only motion compensation in the above- 
described forward direction, but also motion compensation 

10 in backward direction or in both directions of forward and 
backward directions are carried out in combination, thereby 
making it possible to further improve coding efficiency. 

In actual terms, as shown in FIG. 8A, in macro blocks of 
frame data FO, Fl, F2, F3 of the Oth, first, second, third . . 

15 . frames of a video signal of moving picture to be 
transmitted, in the case where there took place changes of 
pictures as respectively indicated by motion vectors xO, xl, 
x2, x3 ♦ . . between frames in succession, device on the 
transmitter side designates frames at intervals of a prede- 

20 termined number of frames (e.g., every other frame), Le., 
second, fourth . . . frames as interpolation frames to imple- 
ment so called predetermined interpolation frame processing 
to these interpolation frames as shown in FIG. 8B to thereby 
generate transmit interpolated frame data F2X, F4X .... 

25 Further, with respect to non-interpolation frames, the device 
on the transmitting side implements a predetermined coding 
processing to frame data Fl, F3 . . . to generate transmit 
non-interpolated frame data FIX, F3X .... 

For example, difference SP2 (predictive error) between 

30 motion compensated frame data F3 and F2, difference SP3 
between motion compensated frame data Fl and F2, and 
difference between frame data obtained by implementing 
interpolation processing to motion compensated frame data 
Fl, F3 and frame data F2 are respectively determined in 

: 35 macro block units to compare difference (data) SP1 of frame 

; data F2 and those differences. Then, data having minimum 

} data quantity generated of those data SP1-SP4 is caused to 

* be transmit interpolated data F2X in macro block units. 
; Similarly, transmit interpolated data F4X . . . with respect to 
f° respective interpolation frames are generated. Further, e.g., 
j DCT processing and variable length coding processing, etc. 
I are implemented to frame data Fl, F3 . . . of non- 
; interpolation frames to generate transmit non-interpolated 

! frame data FIX, F3X 

45 

■ } The transmit non-interpolated frame data FIX, F3X . . . 
[ and transmit interpolated frame data F2X. F4X ... are 
< transmitted to the device on the receiving side as transmit 

data along with motion vectors xO, xl, x3 . . . . 

On the other hand, the device on the receiving side 
I implements decoding processing corresponding to coding 
i processing on the transmitting side to transmit data (transmit 

non-interpolated frame data FIX, F3X . . . , transmit 

interpolated frame data F2X, F4X . . . , data of motion 
; 55 vectors xO, xl, x3 . . . ), thus to reproduce frame data F0, Fl, 
j F2, F3 .... As a result, motion compensation is imple- 
| merited not only in forward direction but also in backward 

direction or in forward and backward directions, thereby 

* making it possible to further improve coding efficiency. 
[6o Picture encoding apparatus and picture decoding appara- 
I tus having the above-described function will now be 
1 described. 

; This picture encoding apparatus comprises, as shown in 
\ FIG. 9, a pre-processing circuit 61 for separating an input 
[65 video signal VD into luminance signal and color difference 
£ signal, analog/digital (hereinafter referred to as A/D) con- 
I verting circuits 62a, 62b for respectively converting the 
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luminance signal and the color difference signal from the 
pre-processing circuit 61 into digital signals, a frame 
memory group 63 for storing luminance data and color 
difference data (hereinafter referred to as picture data) from 
the A/D converting circuits 62a, 626, a format converting 5 
circuit 64 for reading out picture data from the frame 
memory group 63 in accordance with block format, and an 
encoder 65 for implementing efficient coding to picture data 
of block from the format converting circuit 64, 

In operation, pro-processing circuit 61 separates input 10 
video signal VD into luminance signal and color difference 
signal. A/D converting circuits 62a, 626 respectively con- 
verts luminance signal and color difference signal into 
luminance data and color difference data each comprised of 
8 bits. Frame memory group 63 stores these luminance and 15 
color difference data. 

Format converting circuit 64 reads out in accordance 
with block format, picture data (luminance data, color 
difference data) stored in the frame memory group 63. 
Encoder 65 encodes the picture data thus read out by a 20 
predetermined efficient coding to output bit stream. 

This bit stream is delivered to picture decoding apparatus 
80 through transmission media 70 comprised of transmis- 
sion path or picture recording media such as, optical disc, 
magnetic disc or magnetic tape, etc. 25 

This picture decoding apparatus 80 comprises, as shown 
in the FIG. 9 mentioned above, decoder 81 corresponding to 
the encoder 65, format converting circuit 82 for converting 
picture data reproduced by the decoder 81 into frame format, ^ 
frame memory groups 83 for storing picture data from the 
format converting circuit 82, D/A converting circuits 84a, 
846 for converting luminance data, color difference data 
which have been read out from the frame memory group 83 
into analog signals, and post-processing circuit 85 for mix- 35 
ing luminance signal, color difference signal from the D/A 
converting circuits 84a, 846, thus to generate output video 
signal. 

Decoder 81 decodes bit stream by decoding correspond- 
ing to efficient coding of encoder 65 to reproduce picture ^ 
data of block format Format converting circuit 82 converts 
this picture data into frame format to store it into frame 
memory group 83. » 

D/A converting circuits 84a, 846 respectively convert 
luminance data and color difference data which have been 45 
read out from frame memory group 83 into luminance signal 
and color difference signal. Post-processing circuit 81 mixes 
these luminance signal and color difference signat thus to 
generate output video signal. 

In actual terms, pre-processing circuit 61 and A/D con- 50 
verting circuits 62a, 626 convert luminance signal and color 
difference signal into digital signal as described above to 
reduce quantity of data so that the numbers of pixels become 
equal to one half of those of luminance signal in upper and 
lower directions and in left and right directions with respect 55 *~ 
to the luminance signal thereafter to implement time axis , 
multiplexing processing thereto to deliver luminance data 
and color difference data thus obtained to frame memory 
group 63. 

From frame memory group 63, luminance data and color 60 
difference data are read out in accordance with block format 
as described above. Namely, e.g., picture data of one frame 
is divided into N slices as shown in FIG. 10A. Each slice is 
caused to include M macro blocks as shown in FIG. 10B. 
Each macro block is composed of luminance data Yl, Y2, 65 
Y3, Y4 of four luminance blocks consisting of 8x8 pixels 
adjacent in upper and lower directions and in left and right 
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directions and color difference data Cb, Or of color blocks 
consisting of 8x8 pixels in a range corresponding to these 
four luminance blocks. From frame memory group 63, 
luminance data and color difference data are read out so that 
5 picture data are successive in macro block units within slice 
and are successive in order of Yl, Y2, Y3, Y4, Q>, Cr within 
macro block. Picture data which have been read out in 
accordance with block format in this way are delivered to 
encoder 65. 

10 Encoder 65 comprises motion vector detecting circuit 10 1 
as shown in FIG. 11. This motion vector detecting circuit 
101 detects, in macro block units, motion vector of picture 
data delivered thereto in accordance with block format 
Namely, motion vector detecting circuit 101 detects, in 

15 macro block units, motion vector of current reference picture 
by forward original picture and/or backward original picture 
stored in frame memory group 83. Here, detection of motion 
vector is carried out such that minimum one of absolute 
value sums of differences between frames in macro block 

20 units is caused to be corresponding motion vector. The 
motion vector thus detected is delivered to motion compen- 
sating circuit 113, etc., and intra-frame differences in macro 
block units are delivered to intra- frame/forward/backward 
bidirectionally predictive judging circuit 103. 

25 This intra-fxame/forvvard/backward/bidirectionaily pre- 
dictive judging circuit 103 determines predictive mode of 
reference block on the basis of this value to control predic- 
tive coding circuit 104 so as to carry out switching of 
intra-frame/forward^ackward/bidirectional prediction in 

30 macro block units. Predictive coding circuit 104 comprises 
adding circuits 104a, 104&, 104c and selecting (changeover) 
switch 104J. and is operative so that when predictive coding 
mode is intra-frame coding mode, it selects input picture 
itself, and when predictive coding mode is forward/ 

35 backward/bidirectionally predictive mode, it selects differ- 
ences (hereinafter referred to as difference data) every pixels 
of input picture data with respect to respective predictive 
pictures, thus to deliver the selected data to DCT circuit 105. 

^ DCT circuit 105 implements DCT processing to input 
picture data or difference data in block units by making use 
of the two-dimensional correlation of video signal to deliver 
coefficient data thus obtained to quantizing circuit 106. 
The quantizing circuit 108 quantizes coefficient data by 

45 using quantization step size (quantization scale) determined 
every macro block or slice to deliver quantized data thus 
obtained to variable length coding (hereinafter referred to as 
VLC) circuit 107 and inverse quantizing circuit 10B. 
Meanwhile, quantization step size used for this quantization 

5q is determined so as to take a value such that transmitting 
buffer memory 109 which will be described later does not 
break by providing feedback of buffer residual of transmit- 
ting buffer 109. This quantization step size is also delivered 
to VLC circuit 107 and inverse quantizing circuit 10B. 

55 VLC circuit 107 implements variable length coding to 
quantized data along with quantization step size, predictive 
mode and motion vector to deliver them to transmitting 
buffer memory 109 as transmit data. 
The transmitting buffer memory 109 temporarily stores 

60 transmit data thereafter to read out it at a predetermined bit 
rate to thereby smooth transmit data to output it as bit 
stream, and to feed quantization control signal in macro 
block units back to quantizing circuit 108 in accordance with 
residual data quantity remaining in the memory to control 

65 quantization step size. Thus, transmitting buffer memory 
109 adjusts data quantity generated as bit stream to maintain 
data of appropriate residual (remaining capacity) (data quan- 
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tity such that no overflow or underflow takes place) within j 
the memory. For example, when data residual of transmit- j 
ting buffer memory 109 increase to allowed upper limit, j 
transmitting buffer memory 109 allows quantization step 
size of quantizing circuit 108 to be large by quantization 5 
control signal, thus to reduce data quantity of quantized data. 
On the other hand, when data residual of transmit buffer 
memory 109 decrease down to allowed lower limit, trans- 
mitting buffer memory 109 allows quantization step size of 
quantizing circuit 106 to be small by quantization control w , 
signal to thereby increase data quantity. ■ 

In this way, bit stream outputted from buffer memory 109 
is delivered to picture decoded unit 80 through transmission ■ 
media 70 comprised of a transmission path or a picture \ 
recording medium such as optical disc, magnetic disc, or 
magnetic tape etc. at a predetermined bit rate as described 15 
above. 

On the other hand, inverse quantizing circuit 108 inverse- 
quantizes quantized data delivered from quantizing circuit 
106 to reproduce coefficient data (quantization distortion is 
added) corresponding to output of the about-described DCT 20 * 
circuit 105 to deliver the coefficient data to Inverse Discrete 
Cosine Transform (hereinafter referred to as UXT) circuit 
110. 

The EDCT circuit 110 implements 1DCT processing to the 
coefficient data to reproduce picture data corresponding to 
input picture data in the intra-frame coding mode, and to 
reproduce difference data corresponding to output of pre- 
dictive coding circuit 104 in the forward/backward/ 
bidkectionally predictive modes, thus to deliver it to adding ^ 
circuit 111. 

When predictive coding mode is the forward/backward/ 
bidirectionally predictive modes, the adding circuit 111 is 
supplied with motion-compensated predictive picture data 
from motion compensating circuit 113 which will be 35 
described later to add the motion-compensated predictive 
picture data and difference data to thereby reproduce picture 
data corresponding to input picture data. 

The picture data reproduced in this way is stored into 
frame memory 112. Namely, inverse quantizing circuit ^ 
108~adding circuit 111 constitute a local decoding circuit to 
locally decode quantized data outputted from quantizing 
circuit 106 to write decoded picture thus obtained into, .frame 
memory 112 as forward predictive picture or backward 
predictive picture. The frame memory 112 is comprised of a 45 
plurality of frame memories. Bank switching of the frame 
memory is carried out In correspondence with picture to be 
encoded, single frame is outputted as forward predictive 
picture data, or is outputted as backward predictive picture 
data. Moreover, in the case of bidirectional prediction., ^ 
forward predictive picture data and backward predictive 
picture data are. e.g., averaged. The averaged data thus 
obtained is outputted. These predictive picture data are 
entirely the same pictures as pictures reproduced by decoder ^ 
81 which will be described later. Picture to be processed next 55 
is caused to undergo forward/backward/bidirectional predic- 
tive coding on the basis of this predictive picture. 

Namely, picture data which has been read out from frame 
memory 112 is delivered to motion compensating circuit : 
113. This motion compensating circuit 113 implements 
motion compensation to predictive picture data on the basis 
of motion vector to deliver the motion-compensated predic- 
tive picture data to predictive encoding circuit 104 and 
adding circuit 111. 

Decoder 81 will now be described. ^5 

To decoder 8 1, bit stream is inputted through transmission * 
media 70 is inputted. This bit stream is inputted to Variable ' 
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Length Decoding (Inverse Variable Length Coding) 
(hereinafter referred to as IVLC) through receiving buffer 
201, The IVLC circuit 202 reproduces quantized data, 
motion vector, predictive mode and quantization step size, 
etc. from bit stream. These quantized data and quantization 
step size are delivered to inverse quantizing circuit 203. 
Motion vector is delivered to motion compensating circuit 
207, and predictive mode is delivered to adding circuit 205. 

The operation of inverse quantizing circuit 203~adding 
circuit 205 is the same as that of local decoding circuit of 
encoder 61, the operations of frame memory group 206, 
motion compensating circuit 207 are respectively the same 
as those of frame memory 112 and motion compensating 
circuit 113 of encoder 61. On the basis of quantized data, 
motion vector, predictive mode, quantization step size, 
decoding is carried out As a result, reproduction picture data 
is outputted from adding circuit 205. 

As described above, in the conventional apparatus, coding 
bit rate of bit stream generated at encoder 65 is caused to be 
20 fixed in correspondence with transfer rate of transmission 
media 70. Under this limitation, quantity of data generated, 
i.e., quantization step size of quantizing circuit 106 in 
encoder 65 was controlled In other words, for example, a 
control was conducted such that when pictures of compli- 
cated pattern are successive, quantization step size is caused 
to be larger to suppress quantity of data generated, while 
when simple patterns are successive, quantization step size 
is caused to be smaller to increase quantity of data generated 
so that buffer memory 109 does not produce overflow or 
3Q underflow, thus to maintain a fixed rate. 

Accordingly, in the conventional apparatus, when com- 
plicated pictures are successive, quantization step size is 
caused to be larger, so picture quality is deteriorated, while 
when simple pictures are successive, quantization step size 
35 is caused to be smaller. As a result, uniform picture quality 
cannot be obtained through the entirety. 

In addition, in the case of recording bit stream onto a 
picture recording medium of a limited data capacity, in order 
to avoid extreme deterioration of picture quality with respect 
40 to pictures of complicated pattern, a fixed rate of high rate 
such that picture quality of such complicated picture is not 
injured must be applied to the entirety, resulting in decreased 
recording time. 

DISCLOSURE OF THE INVENTION 

45 

In order to solve the above-described problem a first 
picture encoding method according to mis invention com- 
prises the steps of encoding at least a portion of an input 
video signal to generate first encoded data, determining an 
encording rate every predetermined time on the basis of data 
quantity every predetermined time of the first encoded data 
and total quantity of usable data, and encoding the input 
video signal every predetermined time on the basis of the 
encoding rate to generate second encoded data. 

55 A second picture encoding method according to this 
invention is characterized in that in the first picture encod- 
ing method, at least a portion of the input video signal is 
quantized by a fixed quantization step size to generate the 
first encoded data. 

60 A third picture encoding method according to this inven- 
tion is characterized in that, in the first picture encoding 
method, the total quantity of usable data is proportionally 
allocated in accordance with data quantity every predeter- 
mined time, thus to determine the encoding rate every 

65 predetermined time. 

A fourth picture encoding method according to this inven- 
tion comprises the steps of implementing a predetermined 
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predictive encoding or a predetermined transform encoding 
to at least a portion of an input video signal to generate first 
coefficient data, quantizing the first coefficient data by a 
fixed step size to generate first quantized data, allowing the 
first quantized data to undergo variable length encoding to 5 
generate a first bit stream, determining an encoding rate 
every predetermined time on the basis of data quantity of the 
first bit stream and total quantity of usable data, implement- 
ing the predetermined predictive encoding and/or the pre- 
determined transform encoding to the input video signal to 10 
generate second coefficient data, quantizing the second 
coefficient data by a quantization step size based on the 
encoding rate every predetermined time to generate second 
quantized data, and allowing the second quantized data to 
undergo variable length encoding to generate a second bit 15 
stream. 

A fifth picture encoding method according to this inven- 
tion is characterized in that, in the fourth picture encoding 
method, the encoding rate is determined every one frame on 
the basis of data quantity every one frame in the first bit 20 
stream and total quantity of usable data. 

A sixth picture encoding method according to this inven- 
tion is characterized in that, in the fourth pid^e-encoding 
method, the encoding rate is determined every GOP on the 
basis of data quantity of at least a potion every GOP 25 
consisting of a plurality of frames in the first bit stream and 
total quantity of usable data. 

A seventh picture encoding method according to mis 
invention is characterized in that, in the sixth picture encod- 
ing method, encoding rate every GOP is determined on the 30 
basis of data quantity with respect to intra-frame encoded 
picture and forward predictive encoded picture in the GOB 

An eighth picture encoding method according to this 
invention is characterized in that, in the fourth picture 35 
encoding method, the total quantity of usable data is pro- 
portionally allocated in dependency upon data quantity of 
the first bit stream every predetermined time to determine an 
encoding rate every predetermined time. 

A ninth picture encoding method according to this inven- ^ 
tion comprises the steps of determining difficulty of encod- 
ing every predetermined picture unit of an input video 
signal, setting an encoding rate every predetermined picfare 
unit on the basis of the difficulty of encoding and total 
quantity of usable data, and implementing encoding to the 45 
input video signal so that encoding rates of respective 
picture units are in correspondence with the set encoding 
rate every picture unit 

A tenth picture encoding method according to this inven- 
tion is characterized in that, in the ninth picture encoding ^ 
method, the predetermined picture unit is frame. 

An eleventh picture encoding method according to this 
invention is characterized in that, in the ninth picture encod- 
ing method, the predetermined picture unit is GOP consist- 
ing of a plurality of frames. 55 

A twelfth picture encoding method according to this - 
invention is characterized in that in the ninth picture encod- 
ing method, a predetermined predictive encoding and/or a 
predetermined transform encoding is implemented to at least 
a portion of the input video signal to generate coefficient 60 
data to quantize the coefficient data by a fixed quantization 
step size to thereby determine difficulty of encoding. 

A first picture encoding apparatus according to this inven- 
tion comprises first encoding means for encoding at least a 
portion of an input video signal to generate first encoded 65 
data; encoding control means for determining an encoding 
rate every predetermined time on the basis of data quantity 
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every predetermined time of the first encoded data from the 
first encoding means and total quantity of usable data; and 
second encoding means for encoding the input video signal 
every predetermined time on the basis of the encoding rate 

5 every predetermined time from encoding control means to 
generate second encoded data. 

A second picture encoding apparatus according to this 
invention is characterized in that, in the first picture encod- 
ing apparatus, the first encoding means comprises quantiz- 

10 ing means for quantizing at least a portion of the input video 
signal by a fixed quantization step size, 

A third picture encoding apparatus according to this 
invention is characterized in that, in the first picture encod- 
ing apparatus, the encoding control means is operative to 

15 proportionally allocate the total quantity of usable data in 
dependency upon data quantity every predetermined time, 
thus to determine an encoding rate every predetermined 
time. 

A fourth picture encoding apparatus according to this 

20 invention comprises first encoding means for implementing 
a predeterrnined predictive encoding and/or a predetermined 
transform encoding to at least a portion of an input video 
signal to generate first coefficient data; first quantizing 
means for quantizing the first coefficient data from the first 

25 encoding means by a fixed quantization step size to generate 
first quantized data; first variable length encoding means for 
allowing the quantized data from the first quantizing means 
to undergo variable length encoding to generate a first bit 
stream; encoding control means for deteimining an encoding 

30 rate every predeterrnined time on the basis of data quantity 
of the first bit stream from the first variable length encoding 
means and total quantity of usable data; second encoding 
means for implementing the predetermined predictive 
encoding and/or the predictive transform encoding to the 

35 input video signal to generate second coefficient data; sec- 
ond quantizing means for quantizing the second coefficient 
data from the second encoding means by a quantization step 
size based on the encoding rate every predetermined time 
from the encoding control means to generate second quan- 

40 tized data; and second variable length encoding means for 
allowing the second quantized data from the second quan- 
tizing means to undergo variable length encoding to gener- 

^ ate a second bit stream. 

45 A fifth picture encoding apparatus according to this inven- 
tion is characterized in that, in the fourth picture encoding 
apparatus, the encoding control means determines the 
encoding rate every one frame on the basis of data quantity 
every one frame in the first bit stream and total quantity of 

^ usable data. 

A sixth picture encoding apparatus according to this 
invention is characterized in that, in the fourth picture 
encoding apparatus, the encoding control means determines 
the encoding rate every GOP on the basis of data quantity of 

55 at least a portion every GOP consisting of a plurality of 
frames in the first bit stream and total quantity of usable data. 

A seventh picture encoding apparatus according to this 
invention is characterized in that, in the sixth picture encod- 
ing apparatus, the encoding control means determines an 

60 encoding rate every GOP on the basis of data quantity with 
respect to intra-frame encoded picture and forward predic- 
tive encoded picture in the GOP. 

An eighth picture encoding apparatus according to this 
invention is characterized in that, in the fourth picture 

65 encoding apparatus, the encoding control means proportion- 
ally allocate the total quantity of usable data in dependency 
upon data quantity of the first bit stream every predeter- 
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mined time, thus to determine the encoding rate every 
predetermined time. 

A ninth picture encoding apparatus according to this 
invention comprises difficulty calculating means for calcu- 
lating (determining) difficult of encoding every predeter- 5 
mined picture unit of an input video signal; encoding rate 
setting means for setting an encoding rate every predeter- 
mined picture unit on the basis of the difficulty of encoding 
from the difficulty calculating means and total quantity of 
usable data; and encoding means for allowing the input 
video signal to undergo encoding so that encoding rates of 
respective picture units are in correspondence with the 
encoding rate every picture unit set by the encoding rate 
setting means. 

A tenth picture encoding apparatus according to this ^ 
invention is characterized in that, in the ninth picture encod- 
ing apparatus, the difficulty calculating means determines 
difficulty of encoding every frame. 

An eleventh picture encoding apparatus according to this 
invention is characterized in that, in the ninth picture encod- ^ 
ing apparatus, the difficulty calculating means determines 
difficulty of encoding every GOP consisting of a plurality of 
frames. 

A twelfth picture encoding apparatus according to this 
invention is characterized in that in the ninth pictureencod- 25 
ing apparatus, the difficulty calculating means implements a 
predetermined predictive encoding and/or a predetermined 
transform encoding to at least a portion of the input video 
signal to generate coefficient data, and to quantize the 
coefficient data by a fixed step size to thereby calculate 30 
(determine) difficulty of encoding. 

A first picture recording medium according to this inven- 
tion is characterized in that there is recorded a second bit 
stream obtained by encoding at least a portion of an input 
video signal to generate first encoded data to determine an 35 
encoding rate every predetermined time on the basis of data 
quantity every predetermined time of the first encoded data 
and total quantity of usable data to encode the input video 
signal every predetermined time on the basis of the encoding 
rate. 40 

A second picture recording medium according to this 
invention is characterized in that there is recorded a second 
bit stream obtained by implementing a predetermined pre- 
dictive encoding and/or a predetermined transform encoding 
to at least a portion of an input video signal to generate 1kst 45 
coefficient data to quantize the first coefficient data by a 
fixed quantization step size to generate first quantized data 
to allow the first quantized data to undergo variable length 
encoding to generate a first bit stream to determine an 
encoding rate every predetermined time on the basis of data 50 
quantity of the first bit stream and total quantity of usable 
data to implement the predetermined predictive encoding 
and/or the predetermined transform encoding to the input 
video signal to generate second coefficient data to quantize 
the second coefficient data by a quantization step size base$k 55 
on the encoding rate every predetermined time to generate 
second quantized data to allow the second quantized data to 
undergo variable length encoding. 

A third picture recording medium according to this inven- 
tion is characterized in that there is recorded encoded data 60 
obtained by detemiining difficulty of encoding every pre- 
determined picture unit of an input video signal to set 
encoding rate every predetermined picture unit on the basis 
of the difficulty of encoding and total quantity of usable data 
to encode the input video signal so that encoding rates of 65 
respective picture units are in correspondence with the set 
encoding rate every picture unit 



10 

In accordance with the first picture encoding method 
according to this invention* encoding rate every predeter- 
mined time is determined on the basis of data quantity every 
predetermined time of first encoded data obtained by encod- 

5 ing at least a portion of an input video signal and total 
quantity of usable data to encode the input video signal 
every predetermined time on the basis of the encoded rate to 
generate second encoded data. 
In accordance with the second picture encoding method 

10 according to this invention, in the first picture encoding 
method, at least a portion of input video signal is quantized 
by fixed quantization step size to thereby generate the first 
encoded data to determine the encoding rate to encode the 
input video signal every predetermined time on the basis of 

13 the encoding rate to generate second encoded data. 

In accordance with the third picture encoding method 
according to this invention, in the first picture encoding 
method, the total quantity of usable data is proportionally 
allocated in dependency upon data quantity every predeter- 

20 mined time to determine encoding rate every predetermined 
time to encode the input video signal every rffedetermined 
time on the basis of the encoding rate to generate second 
encoded data. 

25 In accordance with the fourth picture encoding method 
according to this invention, predetermined predictive encod- 
ing and/or predetermined transform encoding processing, 
and quantization processing and variable length encoding 
processing at fixed quantization step size are implemented to 

30 at least a portion of an input video signal to generate first bit 
stream to determine encoding rate every predetermined time 
on the basis of data quantity of the first bit stream and total 
quantity of usable data. Then, predetermined predictive 
encoding and/or predetermined transform encoding 

35 processing, and quantization processing and variable length 
encoding processing by quantization step size based on 
encoding rate every predetermined time are implemented to 
input video signal, thus to generate second bit stream. 
In accordance with the fifth picture encoding method 

^ according to this invention, in the fourth picture encoding 
method, the encoding rate is determined every one frame on 
the basis of data quantity every one frame in the first bit 
stream and total quantity of usable data. Then, predeter- 
mined predictive encoding and/or predetermined transform 

45 encoding processing, and quantization processing and vari- 
able length encoding processing at quantization step size 
based on encoding rate every one frame are implemented to 
input video signal, thus to generate second bit stream. 
In accordance with the sixth picture encoding method 

50 according to this invention, in the fourth picture encoding 
method, the encoding rate is determined every GOP on the 
basis of data quantity of at least a portion every GOP 
consisting of a plurality of frames in the first bit stream and 
total quantity of usable data. Then, predeterrnined predictive 

55 encoding and/or predetermined transform encoding 
processing, and quantization processing and variable length 
encoding processing at quantization step size base on encod- 
ing rate every GOP are implemented to input video signal, 
thus to generate second bit stream. 

60 In accordance with the seventh picture encoding method 
according to this invention, in the sixth picture encoding 
method, encoding rate every GOP is determined on the basis 
of data quantity with respect to intra-trame encoded picture 
and forward predictive encoded picture in the GOP. Then, 

65 predetermined predictive encoding and/or predetermined 
transform encoding processing, quantization processing at 
quantization step based on encoding rate every GOP and 
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variable length encoding processing are implemented to 
input video signal, thus to generate second bit stream- 
In accordance with the eighth encoding method according 
to this invention, in the fourth picture encoding method, the 
total quantity of usable data is proportionally allocated in 5 
dependency upon data quantity of the first bit stream eveiy 
predetermined time to determine the encoding rate every 
predetermined time. Then, predetermined predictive encod- 
ing and/or predetermined transform encoding processing, 
quantization processing at quantization step size based on i° 
encoding rate every predetermined time and variable length 
encoding processing are implemented to input video signal, 
thus to generate second bit stream. 

In accordance with the ninth picture encoding method 
according to this invention, difficulty of encoding every 15 
predetermined picture unit of input video signal is deter- 
mined to set encoding rate every predetermined picture unit 
on the basis of the difficulty of encoding and total quantity 
of usable data. Then, the input video signal is encoded so 
that the encoding rates of respective picture units are in 20 
correspondence with the set encoding rate every picture unit 

In accordance with the tenth picture encoding method 
according to this invention, in the ninth picture jenepding 
method, difficulty of encoding is determined every frame of ^ 
input video signal to determine the encoding rate every 
frame. Then, the input video signal is encoded so that 
encoding rates of respective frames are in correspondence 
with the set encoding rate every frame. 

In accordance with the eleventh picture encoding method, ^ 
in the ninth picture encoding method, difficulty of encoding 
is deterrnined every GOP of input video signal to determine 
the encoding rate every GOP. Then, the input video signal is 
encoded so that encoding rates of GOPs are in correspon- 
dence with the set encoding rate every GOR 35 . 

In accordance with the twelfth encoding method accord- ; 
ing to this invention, in the ninth picture encoding method, j 
coefficient data obtained by implementing predetermined j 
predictive encoding and/or predetermined transform encod- • 
ing to at least a portion of input video signal is quantized at ^ j 
fixed quantization step size to thereby detenrine difficulty. \ 
Then, the input video signal is encoded so that encoding j 
rates of respective picture units are in correspondence ^pth \ 
the set encoding rate every picture unit - 1 

Further, in accordance with the first picture encoding 45 
apparatus according to this invention, encoding rate every | 
predetermined time is determined on the basis of data 
quantity every predetermined time of first encoded data 
obtained by encoding at least a portion of input video signal 
and total quantity of usable data to encode the input video 50 
signal every predetermined time on the basis of the encoding 
rate, thus to generate second encoded data. 

In accordance with the second picture encoding apparatus 
according to this invention, in the first picture encoding 
apparatus, at least a portion of input video signal is quan- 55^ 
tized at fixed quantization step size to thereby generate the 
first encoded data to determine the encoding rate to encode 
the input video signal every predetermined time on the basis 
of the encoding rate, thus to generate second encoded data. 

In accordance with the third picture encoding apparatus 60 
according to this invention, in the first picture encoding 
apparatus, the total quantity of usable data is proportionally 
allocated in dependency upon the data quantity every pre- 
determined time to determine the encoding rate every pre- j 
determined time to encode the input video signal every 65; 
predetermined time on the basis of the encoding rate, thus to j 
generate second encoded data. * 
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In accordance with the fourth picture encoding apparatus 
according to this invention, predetermined predictive encod- 
ing and/or predetermined transform encoding processing, 
and quantization processing at fixed quantization step size 

5 and variable length encoding processing are implemented to 
at least a portion of input video signal to generate first bit 
stream to determine encoding rate every predetermined time 
on the basis of data quantity of the first bit stream and total 
quantity of usable data. Then, predetermined predictive 

1Q encoding and/or predetermined transform encoding 
processing, and quantization processing and variable length 
encoding processing at quantization step size based on 
encoding rate every predetermined time, thus to generate 
second bit stream. 

15 In accordance with the fifth picture encoding apparatus 
according to this invention, in the fourth picture encoding 
apparatus, the encoding rate is determined every one frame 
on the basis of data quantity every one frame in the first bit 
stream and total quantity of usable data. Then, predeter- 
mined predictive encoding and/or predictive transform 
processing, and quantization processing at quantization step 
size based on encoding rate every one frame and variable 
length encoding processing are implemented to input video 
signal, thus to generate second bit stream. 

25 In accordance with the sixth picture encoding apparatus 
according to this invention, in the fourth picture encoding 
apparatus, the encoding rate is determined every GOP on the 
basis of data quantity of at least a portion every GOP 
consisting of a plurality of frames in the first bit stream and 

^ total quantity of usable data. Then, predetermined predictive 
encoding and/or predetermined transform encoding 
processing, quantization processing at quantization step size 
based on encoding rate every GOP and variable length 
encoding processing are implemented to input video signal 

35 to generate second bit stream. 

In accordance with the seventh picture encoding appara- 
tus according to this invention, in the sixth picture encoding 
apparatus, the encoding rate every GOP is deterrnined on the 
basis of data quantity with respect to intra-frame encoded 

4Q picture and forward predictive encoded picture in the GOP. 
Then, predetermined predictive encoding and/or predeter- 
mined transform encoding processing, quantization process- 
ing at quantization step size based on encoding rate every 
GOP and variable length encoding processing are imple- 

" 45 mented to input video signal, thus to generate second bit 
stream 

In accordance with the eighth picture encoding apparatus 
according to this invention, in the fourth picture encoding 
apparatus, the total quantity of usable data is proportionally 
5q allocated in dependency upon data quantity of the first bit 
stream every predetermined time to detennine the encoding 
rate every predetermined time. Then, piedetermined predic- 
tive encoding and/or predetermined transform encoding 
processing, quantization processing at quantization step size 
55 based on encoding rate every predetermined time and vari- 
able length encoding processing are implemented to input 
video signal, thus to generate second bit stream. 

In accordance with the ninth picture encoding apparatus 
according to this invention, difficulty of encoding is deter- 
6o rained every predetermined picture unit of input video signal 
to set encoding rate every predetermined picture unit on the 
basis of the difficulty of encoding and total quantity of 
usable data. Then, the input video signal is encoded so that 
encoding rates of respective picture units are in correspon- 
ds dence with the set encoding rate every picture unit. 

In accordance with the tenth picture encoding apparatus 
according to this invention, in the ninth picture encoding 
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apparatus, difficulty of encoding is determined every frame 
of input video signal to determine the encoding rate every 
frame. Then, the input video signal is encoded so that 
encoding rates of respective frames are in correspondence 
with the set encoding rate every frame. 5 

In accordance with the eleventh picture encoding appa- 
ratus according to this invention, in the ninth picture encod- 
ing apparatus, difficulty of encoding is determined every 
GOP of input video signal to determine the encoding rate 
every GOP. Then, the input video signal is encoded so that 10 
encoding rates of respective GOPs are in correspondence 
with the set encoding rate every GOP. 

In accordance with the twelfth picture encoding apparatus 
according to this invention, in the ninth picture encoding 
apparatus, coefficient data obtained by implementing prede- 15 
termined predictive encoding and/or predetennined trans- 
form encoding to at least a portion of input video signal is 
quantized at fixed quantization step size to thereby the 
determine difficulty of encoding. Then, the input video 
signal is encoded so that encoding rates of respective picture 20 , 
units are in correspondence with the set encoding rate every 
picture unit 

In addition, in accordance with the first picture recording 
medium according to this invention, there is recorded sec- ^ 
ond bit stream obtained by determining encoding rate every 
predetermined time on the basis of data quantity every 
predetermined time of first encoded data obtained by encod- 
ing at least a portion of input video signal and total quantity 
of usable data to encode the input video signal every ^ 
predetermined time on the basis of the encoding rate. 

In accordance with the second picture recording medium 
according to this invention, there is recorded second bit 
stream obtained by implementing predetermined predictive 
encoding and/or predetermined transform encoding 35 
processing, and quantization processing and variable length 
encoding processing at fixed quantization step size to gen- 
erate first bit stream to determine encoding rate every 
predetermined time on the basis of data quantity of the first 
bit stream and total quantity of usable data to implement ^ 
predetermined predictive encoding and/or predetermined 
transform encoding processing, and quantization processing 
and variable length encoding processing at quantization st$p 
size based on encoding rate every predetermined time to the 
input video signal. 45 

In accordance with the third picture recording medium 
according to this invention, there is recorded encoded data 
obtained by deterrnining difficulty of encoding every pre- 
determined picture unit of input video signal to set encoding 
rate every predetermined picture unit on the basis of the 50 
difficulty of encoding and total quantity of usable data to 
encode the input video signal so that encoding rates of 
respective picture units are in correspondence with the set 
encoding rate every picture unit. 

BRIEF DESCRIPTION OF THE DRAWINGS 55 

FIG. 1 is a block diagram showing a circuit configuration 
of the essential part of a picture encoding apparatus to which 
mis invention is applied. 

FIG. 2 is a flowchart for explaining the operation of first ^ 
encoding circuit constituting the above-mentioned picture 
encoding apparatus. 

FIG. 3 is a flowchart for explaining the operation of 
second encoding circuit constituting the above-mentioned 
picture encoding apparatus. 65 

HG. 4 is a view showing respective pictures for explain- 
ing the configuration of GOP in MPEG. 
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FIG. 5 is a view showing respective pictures for explain- 
ing encoding control signal every GOP. 

FIG. 6 is a flowchart for explaining the operation of the 
second encoding circuit constituting 'the above-mentioned 
5 picture encoding apparatus. 

FIG. 7 is a view showing picture for explaining the 
principle of predictive encoding. 

FIG. 8 is a view showing picture for explaining the 
10 principle of motion compensated predictive encoding. 

FIG. 9 is a block diagram showing the configuration of 
picture encoding apparatus (unit) and picture decoding appa- 
ratus (unit) which are related to this invention. 

FIG. 10 is a view showing the configuration of frame, 
15 macro block and slice. 

FIG. 11 is a block diagram showing a circuit configuration 
of conventional encoder. 

FIG. 12 is a block diagram showing a circuit configuration 
of conventional decoder. 

20 

BEST MODE FOR CARRYING OUT THE 
INVENTION 

An embodiment of a picture encoding method, a picture 

25 encoding apparatus and a picture recording medium accord- 
ing to this invention will now be described with reference to 
the attached drawings. 

A picture encoding apparatus to which this invention is 
applied comprises, as shown in FIG. I, for example, a first 

30 encoding circuit 10 for encoding an input video signal to 
generate first encoded data, an encoding control circuit 30 
for determining encoding rate every predetermined time on 
the basis of data quantity every predetermined time of the 
first encoded data from the first encoding circuit 10 and total 

35 quantity of usable data, and a second encoding circuit 40 for 
encoding the input video signal every predetermined time on 
the basis of the encoding rate from the encoding control 
circuit 30 to generate second encoded data. 

More particularly, the first encoding circuit 10 comprises, 

40 as shown in the FIG. 1 mentioned above, a frame memory 
group 12 for storing input picture data which is input video 
signal, a motion vector detecting circuit 11 for detecting 
motion vector of input picture data from input picture data 
" on the basis of picture data stored in the frame memory 

45 group 12, a frame memory 22 for storing predictive picture 
data, a motion compensating circuit 23 for implementing 
motion compensation to predictive picture data which has 
been read out from the frame memory 22 on the basis of 
motion vector from the motion vector detecting circuit 11, a 

50 predictive encoding circuit 14 for predictive-encoding input 
picture data on the basis of motion-compensated predictive 
picture data from the motion compensating circuit 23, a 
DCT circuit 15 for implementing encoding, e.g., Discrete 
Cosine Transform (hereinafter referred to as DCT) process- 

55 ing to differences, etc which are predictive errors from the 
predictive encoding circuit 14 to generate coefficient data, a 
quantizing circuit 18 for quantizing coefficient data from the 
DCT circuit 15 at a fixed quantization step size to generate 
quantized data, a Variable Length Code (hereinafter referred 

60 to as VLC) circuit 17 for allowing quantized data from the 
quantizing circuit 18 to undergo variable length encoding to 
output variable length encoded data, an inverse quantizing 
circuit 18 for inverse-quantizing quantized data from the 
quantizing circuit 16 to reproduce coefficient data, an 

65 Inverse Discrete Cosine Transform (hereinafter referred to 
as IDCT) circuit 20 for implementing decoding, e.g., IDCT 
processing to coefficient data from the inverse quantizing 
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circuit 18, and an adding circuit 21 for adding difference 
from the TDCT circuit 20 and motion-compensated predic- 
tive picture data from the motion compensating circuit 23 to 
generate predictive picture data with respect to the next 
input picture data to deliver the predictive picture data to the 5 
frame memory 22. 

Moreover, the second encoding circuit 40 comprises, as 
shown in the FIG. 1 mentioned above, a delay element 43 for 
delaying input picture data, a frame memory 52 for storing 
predictive picture data, a motion compensating circuit 53 for 
implementing motion compensation to predictive picture 10 
data which has been read out from the frame memory 52 on 
the basis of motion vector from the motion vector detecting 
circuit 11, a predictive encoding circuit 44 for predictive- 
encoding input picture data delayed at the delay element 43 
on the basis of the motion compensated predictive picture 15 
data from the motion compensating circuit 53, a DCT circuit 

45 for implementing encoding, e.g., DCT processing to 
difference, etc. from the predictive encoding circuit 44 to 
generate coefficient data, a quantization scale setting circuit 
33 for setting quantization step size on the basis of encoding 20 
rate from the encoding control circuit 30, a quantizing circuit 

46 for quantizing coefficient data from the DCT circuit 45 at 
quantization step size from the quantization scale setting - 
circuit 33, a VLC circuit for allowing quantized data from 
the quantizing circuit 46 to undergo variable length encod- 25 
ing to output variable length encoded data, a transmitting 
buffer memory 49 for temporarily storing variable length 
encoded data from the VLC circuit 47 to output it at a fixed 
bit rate, an inverse quantizing circuit 48 for inverse- 
quantizing quantized data from the quantizing circuit 46 to 30 
reproduce coefficient data, an IDCT circuit 50 for imple- 
menting decoding, e.g., IDCT processing to the coefficient 
data from the inverse quantizing circuit 48 to reproduce the 
difference, and an adding circuit 51 for adding difference 
from the IDCT circuit 50 and the motion-compensated 35 
predictive picture data from the motion compensating circuit 
53 to generate predictive picture data with respect to the next 
input picture data to deliver the predictive picture data to the 
frame memory 52, 

In this picture encoding apparatus, by the first encoding 40 
circuit 10, encoding processing, e.g., predictive encoding 
processing, DCT processing, quantization processing at . 
fixed quantization step size and variable length encoding * 
processing are implemented to input picture data. By encod- 
ing control circuit 30, encoding bit rate is determined on the 45 
basis of data quantity every predetermined time of variable 
length encoded data which is the first bit stream obtained and 
data capacity of picture recording medium 55, e.g., com- 
prised of optical disc, magnetic disc or magnetic tape, etc. or 
total quantity of usable data determined by bit rate of 50 
transmission path (transfer rate). Thereafter, by second 
encoding circuit 40, predictive encoding processing, DCT 
processing, quantization processing and variable length 
encoding processing are implemented to input picture data 
for a second time. In generating variable length encoded data 55 
which is second bit stream, quantization is made by quan- 
tization step size based on encoding bit rate. 

Namely, in this picture encoding apparatus, as shown in 
FIG. 2, for example, at step ST1, quantizing circuit 16 of 
first encoding circuit 10 sets quantization step size to, e.g., 60" 
1 to quantize coefficient data delivered from DCT circuit 15 
to generate quantized data. Counter 31 of encoding control , 
circuit 30 counts data quantity of variable length encoded > 
data (first bit stream) obtained by allowing the quantized j 
data to undergo variable length encoding every predeter- 65 
mined time, e.g., one frame to determine, every frame, 
quantity of code generated indicating difficulty of encoding. 
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At step ST2, bit rate calculating circuit 32 determines 
allocated code quantity allocated every frame on the basis of 
difficulty (quantity of code generated) every frame and total 
quantity of usable data. 
5 At step ST3, quantizing circuit 46 of second encoding 
circuit 40 quantizes coefficient data delivered from OCT 
circuit 45 by quantization step size based on allocated code 
quantity to generate quantized data. 

In actual terms, inputted picture data is temporarily stored 
10 into frame memory group 12. From the frame memory group 
12, mat picture data is read out in accordance with block 
format as described in the prior art 

Motion vector detecting circuit 11 reads out necessary 
picture data from frame memory group 12 in macro block 
15 units described above to detect motion vector. Namely, 
motion vector detecting circuit 11 detects, in macro block 
units, motion vector of current reference picture by using 
forward original picture and/or backward original picture 
stored in frame memory group 12. Here, detection of motion 
20 vector is carried out such that a motion vector in which, e.g., 
absolute value sum of differences between frames in macro 
block units becomes minimum is caused to be corresponding 
motion vector. The detected motion vector is delivered to 
motion compensating circuits 23, 53, etc., and absolute 
25 value sum of differences between frames in macro block 
units is delivered to intra-frame/forward/backward/ 
bidirectionally predictive judging circuit 13. 

The intra-frame/forward/backward/bidirectionally predic- 
tive judging circuit 13 determines predictive mode on the 
30 basis of the above value to control predictive encoding 
circuit 14 so as to carry out switching of intra -frame/ 
forward/backward/bidirectional prediction in block units. 
The predictive encoding circuit 14 comprises, as shown in 
35 FIG. 1 mentioned above, adding circuits 14<2, 14Z>, 14c and 
selecting (changeover) switch 14^. When predictive encod- 
ing mode is intra-frame encoding mode, the selecting switch 
I4d select input picture data itself, and when predictive 
encoding mode is forward/backwardTbidfrectionally predic- 
^ tive mode, it selects differences (hereinafter referred to as 
difference data) every pixels of input picture data with 
respect to respective predictive pictures. Then, the selecting 
switch I4d delivers selected data to DCT circuit 15. 
The DCT circuit 15 implements, in block units, DCT 
45 processing to input picture data or difference data delivered 
from selecting switch 14d by making use of the two- 
dimensional correlation of video signal to deliver coefficient 
data thus obtained to quantizing circuit 16. 

The quantizing circuit 16 quantizes coefficient data deliv- 
50 ered from DCT circuit 15 at a fixed quantization step size, 
e.g., with quantization step size being set to 1 to deliver 
quantized data thus obtained to VLC circuit 17 and inverse 
quantizing circuit 18. 

The VLC circuit 17 carries out variable length encoding 
55 of quantized data along with quantization step size, predic- 
tive mode, and motion vector, etc. to deliver variable length 
encoded data obtained to encoding control circuit 30 as a 
first bit stream. 

The encoding control circuit 30 comprises, as shown in 
60 the FIG. 1 mentioned above, counter 31 for counting data 
quantity every predetermined time of variable length 
encoded data from the VLC circuit 17, and bit rate calcu- 
lating circuit 32 for calculating (determining) allocated code 
quantity per unit time on the basis of data quantity from the 
65 counter 31 and total quantity of usable data. The counter 31 
counts data quantity of the first bit stream every predeter- 
mined time, e.g., every one frame to determine (calculates) 
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difficulty every frame to deliver this difficulty to bit rate 
calculating circuit 32. 

The bit rate calculating circuit 32 calculates (determines) 
allocated code quantity allocated every frame, Le., mean 
encoding rate every frame time on the basis of difficulty 
every frame and total quantify of usable data, and delivers 
this allocated code quantity to quantization scale setting 
circuit 33 of second encoding circuit 40. 

In actual terms, bit rate calculating circuit 32 performs the 
following calculation. Namely, assuming now mat the num- 
ber of all frames is N, total quantity of usable data is B, 
difficulty of the i (i=0, 1, 2, . . . N-l)-th frame is d,-, and 
allocated code quantity with respect to the i-th frame is b^ 
when this allocated code quantity b t is caused to be propor- 
tional to difficulty df as indicated by the following formula 15 
(1), data total quantity B can be calculated by adding 
allocated code quantities b- of all frames as indicated by the 
following formula (2). In the formula, a represents constant 

br*xd t (i) ^ 

N-i (2) 

B = X 6; 

N-l N-1 - : 

= X axdi = ax X 

" r=0 i=0 ' 25 

Accordingly, constant a can be calculated by the follow- 
ing formula (3). When substitution of this constant a into the 
formula (1) is made, allocated code quantity b ( - with respect 
to the i-th frame can be calculated by the following formula 
(4). 
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(3) 



N-l 

x 4 

i=0 
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Thus, bit rate calculating circuit 32 increases allocated 
code quantity b f with respect to frame of picture of compli- 40 
cated pattern, for example, and decreases allocated code 
quantity b ( with respect to frame of simple pattern. ^ 

On the other hand, inverse quantizing circuit 18 inverse- 
quantizes quantized data delivered from quantMng'circuit 
16 at quantization step size caused to be set to 1 to reproduce 45 
coefficient data (quantization distortion is added) corre- 
sponding to output of DCT circuit 15 to deliver that coef- 
ficient data to IDCT circuit 20. 

The IDCT circuit 20 implements IDCT processing to 
coefficient data to reproduce input picture data correspond- 50 
ing to output of predictive encoding circuit 14 in the 
intra-frame encoding mode, and reproduces difference data 
in the foward/backward/bidirectionally predictive mode, 
thus to deliver reproduced data to adding circuit 21. 

The adding circuit 21 is supplied, when predictive encod-**55 
ing mode is forward/backward/bidirectionally predictive 
mode, with motion-compensated predictive picture data 
from morion compensating circuit 23. The adding circuit 21 
adds this predictive picture data and difference data deliv- 
ered from IDCT circuit 20 to thereby reproduce picture data 60 
corresponding to input picture data. 

The picture data reproduced in this way is stored into 
frame memory 22 as predictive picture data. Namely, 
inverse quantizing circuit 18~adcling circuit 21 constitute a 
local decoding circuit to locally decode quantized data 65 
outputted from quantizing circuit 18 on the basis of predic- 
tive mode to write decoded picture obtained into frame 
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memory 22 as forward predictive picture or backward 
predictive picture. Frame memory 22 is composed of a 
plurality of frame memories. Bank switching of frame 
memory is carried out In dependency upon picture to be 

5 encoded, e.g., single frame is outputted as forward predic- 
tive picture data, or is outputted as backward predictive 
picture data. Further, in the case of forward/backward/ 
bidirectional prediction, forward predictive picture data and 
backward predictive picture data are, e.g.„ averaged and the 

10 averaged data is outputted. These predictive picture data are 
entirely the same picture data as picture data reproduced by 
picture decoding apparatus which will be described later. 
Picture to be processed next is caused to undergo forward/ 
backward/bidirectionally encoding on the basis of this pre- 
dictive picture. 

15 The operation of second encoding circuit 40 will now be 
described It is to be noted that since circuits except for 
quantization scale setting circuit 33, delay element 43, 
quantizing circuit 48, and transmitting buffer 49 constituting 
second encoding circuit 40 perform the same operations as 

20 those of circuits constituting the above-described first 
encoding circuit 10, their explanation is omitted. 

Delay element 43 delays input picture data, e.g., by time 
until encoding control signal is outputted from encoding 
control circuit 30. Then, at predictive encoding circuit 44 

25 and DCT circuit 45, predictive encoding processing and 
DCT processing which are in accordance with predictive 
mode delivered from intra-frame/forward/backward/ 
bidirectionally predictive judging circuit 13 are imple- 
mented to the delayed input picture data. Thus, coefficient 

30 data is generated. 

Quantization scale setting circuit 33 determines 
(calculates) allocated code quantity every macro block (e.g., 
value obtained by dividing allocated code quantity every 
frame by the number of macro blocks in one frame) from 

35 delivered allocated code quantity every frame to carry out 
comparison between code quantity generated in a macro 
block which is detected from buffer feedback from trans- 
mitting buffer 49 and allocated code quantity every macro 
block. The quantization scale setting circuit 33 operates as 

40 follows so as to allow encoding bit rates of respective frames 
to become close to set mean encoding bit rate every frame 
time. Namely, in the case where code quantity generated in 
corresponding macro block is greater than allocated code 
quantity every macro block, the circuit 33 sets quantization 

45 step size of next macro block to a greater value in order to 
suppress code quantity generated by the next macro block. 
In contrast, in the case where code quantity generated in 
corresponding macro block is smaller than allocated code 
quantity every macro block, the circuit 33 sets quantization 

50 step size of next macro block to a smaller value so as to 
increase code quantity generated. It should be noted that 
quantization scale setting circuit 33 is operative so that in the 
case where buffer feedback from transmitting buffer 49 
indicates that transmitting buffer 49 is in a state close to 

55 overflow state, it allows quantization step size to be larger to 
suppress overflow irrespective of comparison result between 
the allocated code quantity and code quantity generated, 
while in the case where buffer feedback from the transmit- 
ting buffer 49 indicates that transmitting buffer 49 is in a 

60 state close to underflow state, it allows quantization step to 
be smaller to suppress underflow irrespective of comparison 
result between the allocated code quantity and code quantity 
generated. While it has been described that comparison 
between code quantity generated and allocated code quantity 

65 is made every macro block to switch quantization step size 
every macro block, such switch may be carried out every 
slice. While it has been described that code quantity gener- 
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ated is detected from storage quantity of transmitting buffer 
49, it may be directly obtained from output of variable 
length encoding circuit 47. The quantization scale setting 
circuit 33 delivers quantization step size set in mis way to 
quantizing circuit 46. 5 

The quantizing circuit 46 quantizes coefficient data deliv- 
ered from DCT circuit 45 by quantization step size delivered 
from the above-described quantization scale setting circuit 
33 to generate quantized data. 

VLC circuit 47 allows quantized data delivered from 10 
quantizing circuit 46 to undergo variable length encoding 
along with quantization step size from quantization scale 
setting circuit 33, predictive mode from intra-frame/ 
forward/backward/bidirectionaliy predictive judging circuit 
13, and motion vector from motion vector detecting circuit is 
11, etc. to deliver variable length encoded data obtained to 
transmitting buffer memory 49 as a second bit stream. 

Namely, in this picture encoding apparatus, as shown in 
FIG. 3, for example, when picture data is inputted through 
delay element 43 at step ST1, quantization scale setting 20 
circuit 33 reads in, from encoding control circuit 30, allo- 
cated code quantity with respect to frame to be currently 
encoded at step ST2, then, the processing operation proceeds 
to step ST3. 

At step ST3, predictive encoding circuit 44-VLC circuit 25 
47 implement predictive encoding processing and DCT 
processing to picture data, and quantizes coefficient data by 
quantization step size based on allocated code quantity 
thereafter to allow it to undergo variable length encoding. 
Then, the processing operation proceeds to step ST4. 30 

At the step ST4, whether or not encoding processing has 
been completed with respect to all frames (sequence) to 
which, e.g., the same picture size or the same transfer rate is 
applied is judged. If so, the processing is completed. In 
contrast, if not so. the processing operation returns to step 35 
STL Thus, variable rate encoding such that encoding rate 
changes in frame units is realized. Accordingly, even if 
pictures (frames) of complicated pattern are successive, 
there is no possibility that quantization step size is caused to 
be large with respect to these pictures as in the conventional 40 
apparatus. Thus, uniform high picture "quality can be 
obtained through the entirety. 

The transmitting buffer memory 49 temporarily stores 
variable length encoded data thereafter to read out it at a 
fixed bit rate to thereby smooth the variable length encoded 45 
data to output it as bit stream. The bit stream which has been 
outputted from transmitting buffer 49 is multiplexed along 
with, e.g., encoded audio signal, synchronizing signal, etc. 
Further, code for error correction is added thereto, and a 
predetermined modulation suitable for transmission or 50 
recording is applied thereto. Thereafter, bit stream thus 
processed is transmitted to picture decoding apparatus 
through, e.g., transmission path, or is recorded onto picture 
recording medium 55 comprised of optical disc, magnetic 
disc or magnetic tape, etc. as shown in the FIG. 1 mentionecL 55 
above. Namely, since, in the second encoding circuit 40, 
there is carried out a variable rate encoding such that e.g., 
allocated code quantity b £ is increased in advance with 
respect to complicated picture and allocated code quantity b t 
is decreased with respect to simple picture, there is no 60 
necessity of applying a fixed rate of high rate through the 
entirety in order to avoid extreme deterioration of picture 
quality with respect to pictures of complicated pattern as in 
the case of conventional apparatus. Thus, recording time of 
picture recording medium 55 can be prolonged. 65 

On the other hand, inverse quantizing circuit 48 inverse- 
quantizes quantized data delivered from quantizing circuit 
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46 by quantization step size used in the above-described 
quantizing circuit 46 to reproduce coefficient data 
(quantization distortion is added) corresponding to output of 
DCT circuit 45 to deliver this coefficient data to IDCT circuit 

5 50. Namely, inverse quantizing circuit 4$~adding circuit 51 
constituting a local decoding circuit locally decode quan- 
tized data outputted from quantizing circuit 46 to write 
decoded picture obtained into frame memory 52 as forward 
predictive picture or backward predictive picture. Picture 

10 data stored in frame memory 52 is used as predictive picture 
for picture to be processed next 

Meanwhile, while, in the above-described embodiment, 
allocated code quantity per predetermined time, i.e., mean 
encoding rate per predetermined time is obtained every 

15 frame with frame being used as predetermined time, this 
invention is not limited to such an implementation. For 
example, GOP (Group of Pictures in so called MPEG 
(Moving Picture Expert Group) may be used as a predeter- 
mined time. It should be noted that the above-described 

20 MPEG is general name of the moving picture encoding 
system being studied in WG (Working Group) 11 of SC (Sub 
Committee) 29 in JTC (Joint Technical Committee) of so 
called ISO (International Standardization Organization) and 
DEC (International Electrotechnical Committee). 

25 Namely, GOP in MPEG consists of at least one so called 
I picture, and a plurality of P pictures or B pictures (non-I 
picture). In a more practical sense, assuming that GOP 
consists of a single I picture, four P pictures having a period 
of 3 pictures, and ten B pictures, encoding control circuit 30 

30 determines allocated code quantity every GOP. Here, I 
picture is picture to be subjected to intra-field or intra-frame 
encoding. P picture is picture which can be predicted only 
from forward direction, and is subjected to interfleld or 
intra-frame encoding. B picture is picture which can be 

35 predicted from forward direction, from back ward direction 
and from both directions and is subjected to interfleld or 
intra-frame encoding. 

When it is assumed, as shown in FIG. 5, for example, that 
successive arbitrary two pictures within GOP having the 

40 number of pictures constituting GOP as period are I picture 
and P picture, and quantization step size is, e,g., 1, the first 
encoding circuit 10 implements predictive encoding 
processing, DCT processing, and variable length encoding 
processing to picture data of these I and P pictures to 

45 generate variable length encoded data to deliver the variable 
length encoded data to encoding control circuit 30. The 
reason why two pictures are used as I picture, P picture is to 
examine complexity of pattern and correlation between 
frames. From code quantity generated of I picture, it is 

50 possible to recognize complexity of pattern. From code 
quantity generated of P picture, it is possible to recognize 
correlation between frames. Since successive plural frames 
have similar pictorial images in general, it is possible to 
recognize tendency of pattern of GOP even from extracted 

55 two pictures. 

Encoding control circuit 30 counts (calculates), every 
GOP, data quantity of I picture bit I y and data quantity of P 
picture bit P^, and determines, every GOP, difficulty (code 
quantity generated GOP d y (j=0, 1, 2 . . . ) on the basis of 

60 these data quantities bit 1^, bit P,- and the number N of P 
pictures constituting GOP as indicated by the following 
formula (5), for example. 

GOPd^yitlj+NxbitPj (5) 

65 The encoding control circuit 30 determines allocated code 
quantity allocated every GOP on the basis of difficulty (code 
quantity generated) GOP d y - every GOP and total quantity of 
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usable data, and delivers this allocated code quantity to 
second encoding circuit 40. 

In actual terms, when the number of GOPs is assumed to 
be M, total quantity of usable data is assumed to be B, 
allocated code quantity with respect to the j-th GOP is GOP 5 
b r and this allocated code quantity GOP b^ is caused to be 
in proportion to difficulty as indicated by the following 
formula (6), data total quantity B is determined by adding 
allocated code quantities GOP b y of all GOPs as indicated by 
the following formula (7). In the formula (6), a is constant 

GOPbj=axGOPdj (6) 10 

M-i 
B= X GOPbi 
J=0 

15 

Accordingly, constant a can be determined By the fol- 
lowing formula (8). Substituting this constant a into the 
formula (6), 



B= X axGOPd^aX X GOPd; 20 

allocated code quantity GOPb^ with respect to the j-th GOP 
can be determined by the following formula 



X GOPdj 



GOPbj = GOPdjX 



Thus, encoding control circuit 30 increases allocated code 
quantity GOP b y - with respect to, eg., GOP in which pictures 
of complicated pattern are included or having low correla- 
tion between frames, and decreases allocated code quantity 35 
GOP b^ with respect to GOP in which pictures of simple 
pattern are included or having high correlation between 
frames. 

When second encoding circuit 40 is supplied with picture 
data through delay element 43 at step ST1, as shown in FIG. 
6, for example, it judges, at step ST2, whether or not picture 40 
data being inputted is the leading picture of GOP If so, the 
processing operation proceeds to step ST3. If not £>, the 
processing operation proceeds to step ST4. 

At step ST3, second encoding circuit 40 reads in allocated 
code quantity with respect to GOP currently subjected to 4 5 
encoding from encoding control circuit 30. Then, the pro- 
cessing operation proceeds to step ST4. 

At the step ST4, second encoding circuit 40 implements 
predictive encoding processing and OCT processing, and 
quantizes coefficient data by quantization step size based on 50 
allocated code quantity thereafter to allow it to undergo 
variable length encoding. Then, the processing operation 
proceeds to step ST5. 

Here, quantization scale setting circuit 33 sets allocated 
code quantity every frame from delivered allocated code ^ 
quantity every GOP by taking into consideration picture type 
(I picture, P picture, B picture) in actual encoding, e.g.,' 
picture type shown in FIG. 4. In a more practical sense, 
allocated code quantity with respect to I picture is increased, 
allocated code quantity with respect to B picture is 
decreased, and allocated code quantity with respect to P 60 
picture is caused to be intermediate therebetween. 

At the subsequent step ST5, whether or not encoding 
processing has been completed with respect to all frames 
(sequence) to which the same picture size or the same 
transfer rate is applied is judged. If so, the processing is 65 
completed In contrast, if not so, the processing operation 
returns to step ST1. Thus, a variable rate encoding such that 
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encoding rate changes every GOP is realized. Even if 
pictures (frames) of complicated pattern are successive, 
there is no possibility that quantization step size is caused to 
be large with respect to these pictures as in the conventional 
apparatus. As a result, it is possible to obtain uniform high 
picture quality over the entirety. Further, since allocated 
code quantity every GOP is determined on the basis of two 
pictures in this embodiment higher speed processing can be 
carried out as compared to the above-described embodi- 
ment It is to be noted that it is a matter of course to 
determine allocated code quantities of respective GOPs on 
the basis of data quantities of all pictures within GOP. 

It should be noted that this invention is not limited to the 
above-described embodiments. While, e.g., transform cod- 
ing employs DCT in the above-described embodiments, so 
15 called Strato transform, Haar transform or Wavelet 
transform, etc. may be employed. 

Industrial Applicability 

As is clear from the foregoing description, this invention 

20 employs a scheme to allow an input video signal to undergo 
encoding, e.g., predictive encoding, DCT processing, quan- 
tization at fixed quantization step size and variable length 
encoding to generate first encoded data to determine 
(calculate) allocated code quantity every frame or every 

25 GOP on the basis of data quantity every predetermined time, 
e.g., every frame or every GOP of the first encoded data and 
total quantity of usable data to encode the input video signal 
every predetermined time on the basis of the allocated code 
quantity to generate second encoded data. Thus, variable 

M rate encoding such that encoding rate changes every prede- 
tenriined time is realized. As a result, even if pictures 
(frames) of complicated are successive, there is no possi- 
bility that quantization step size is caused to be large with 
respect to these pictures as in the conventional apparatus. 
Thus, uniform high picture quality can be obtained over the 

35 entirety. 

Further, since second encoded data obtained in a manner 
as described above has variable rate, in the case where such 
encoded data is recorded onto picture recording media, 
limited memory capacity can be effectively used, and 
40 recording time of picture recording media can be prolonged. 
In addition, picture data of high picture quality uniform 
through the entirety can be reproduced from the picture 
recording media. 
What is claimed is: 
45 J^An encoding method, comprising the steps of: 
IS receiving an input video signal; 

selectively encoding at least a portion of said input video 
signal by intra-frame encoding or predictive encoding 
to generate first encoded data, said predictive encoding 
50 including forward predictive encoding and backward 
predictive encoding; 
transform encoding said first encoded data to generate 
first coefficient data; 
55 quantizing said first coefficient data by a fixed step size; 
variable length encoding said first quantized data to 

generate a first bit stream; 
detenruning an encoding rate of said first bit stream every 
GOP as a function of a data quantity of intra-frame and 
60 forward-predictive encoded pictures only in the GOP; 
selectively encoding said input video signal by intra- 
frame or predictive encoding to generate second 
encoded data; 

transform encoding said second encoded data to generate 
65 second coefficient data; 

setting a variable step size according to the encoding rate 
determined every GOP, 
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quantizing said second coefficient data by said variable 
step size; 

variable length encoding said second quantized data to 
generate a second bit stream; and 

outputting said second bit stream, 5 

2yAn encoding apparatus, comprising: 
'means for receiving an input video signal; 

means for selectively encoding at least a portion of said 
input video signal by intra-frame encoding or predic- 
live encoding to generate first encoded data, said pre- 
dictive encoding including forward predictive encoding 
and backward predictive encoding; 

means for transform encoding said first encoded data to 
generate first coefficient data; ^ 

means for quantizing said first coefficient data by a fixed 
step size; 

means for variable length encoding said first quantized 

data to generate a first bit stream; 
means for determining an encoding rate of said first bit ^ 

stream every GOP as a function of a data quantity of 

intra-frame and forward-predictive encoded pictures 

only in the GOP; 
means for selectively encoding said input video signal by 

intra-frame or predictive encoding to generate second ^ 

encoded data; 

means for transform encoding said second encoded data 
to generate second coefficient data; 

means for setting a variable step size according to the 
encoding rate determined every GOP; 30 

means for quantizing said second coefficient data by said 
variable step size; 

means for variable length encoding said second quantized 
data to generate a second bit stream; and 

means for outputting said second bit stream 35 

3. A recording medium on which there is recorded a 
seperad bit stream obtained by: 

receiving an input video signal; 

selectively encoding at least a portion of said input video 
signal by intra-frame encoding or predictive encoding 40 
to generate first encoded data, said predictive encoding 
including forward predictive encoding and backward ^ 
predictive encoding; 

transform encoding said first encoded data to generate 
first coefficient data; 45 

quantizing said first coefficient data by a fixed step size; 

variable length encoding said first quantized data to 
generate a first bit stream; 

determining an encoding rate of said first bit stream every ^ 
GOP as a function of a data quantity of intra-frame and 
forward-predictive encoded pictures only in the GOP, 

selectively encoding said input video signal by intra- 
frame or predictive encoding to generate second 
encoded data; 55 

transform encoding said second encoded data to generate 
second coefficient data; 

setting a variable step size according to the encoded data 
rate determined every GOP; 

quantizing said second coefficient data by said variable 60 
step size; 

variable length encoding said second quantized data to 

generate a second bit stream; and 
recording said second bit stream on said recording 

medium. 65 
^jf. An encoding method, comprising the steps of: 
receiving an input video signal; 
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selectively encoding at least a portion of said input video 
signal by inter-frame encoding or predictive encoding 
to generate first encoded data, said predictive encoding 
including forward predictive encoding and backward 
5 predictive encoding; 

transform encoding said first encoded data to generate 
first coefficient data; 

quantizing said first coefficient data by a fixed step size; 

variable length encoding said first quantized data to 
10 generate a first bit stream; 

determining a difficulty of encoding said first bit stream 
every GOP based on inter-frame and forward- 
predictive encoded pictures only in the GOP; 

calculating an encoding rate from said difficulty of encod- 
15 ing determined every GOP; 

selectively encoding said input video signal by intra- 
frame or predictive encoding to generate second 
encoded data; 

transform encoding said second encoded data to generate 
20 second coefficient data; 

quantizing the second coefficient data by a step size set 

according to said calculated encoding rate; 
variable length encoding said second quantized data to 
generate a second bit stream; and 
25 outputting said second bit stream. 

5^ An encoding apparatus, comprising: 
^means for receiving an input video signal; 

means for selectively encoding at least a portion of said 
^ input video signal by intra-frame encoding or predic- 
tive encoding to generate first encoded data, said pre- 
dictive encoding including forward predictive encoding 
and backward predictive encoding; 
means for transform encoding said first encoded data to 
35 generate first coefficient data; 

means for quantizing said first coefficient data by a fixed 
step size; 

means for variable length encoding said first quantized 

data to generate a first bit stream; 
^ means for determining a difficulty of encoding said first 

bit stream every GOP based on intra-frame and. 

forward-predictive encoded pictures only in the GOP; 
means for calculating an encoding rate from said difficulty 

of encoding determined every GOP; 
45 means for selectively encoding said input video signal by 

intra-frame or predictive encoding to generate second 

encoded data; 

means for transform encoding said second encoded data 
to generate second coefficient data; 
5q means for quantizing the second coefficient data by a step 
size set according to said calculated encoding rate; 
means for variable length encoding said second quantized 

data to generate a second bit stream; and 
means for outputting said second bit stream. 
55 <y A recording medium on which there is recorded a 
second bit stream obtained by: 
receiving an input video signal; 

selectively encoding at least a portion of said input video 

signal by intra-frame encoding or predictive encoding 
60 to generate first encoded data, said predictive encoding 

including forward predictive encoding and backward 

predictive encoding; 
transform encoding said first encoded data to generate 

first coefficient data; 
65 quantizing said first coefficient data by a fixed step size; 
variable length encoding said first quantized data to 

generate a first bit stream; 
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determining a difficulty of encoding said first bit stream 
every GOP based on intra-frame and forward- 
predictive encoded pictures only in the GOP; 

calculating an encoding rate from said difficulty of encod- 
ing determined every GOP; 5 

selectively encoding said input video signal by intra- 
frame or predictive encoding to generate second 
encoded data; 

transform encoding said second encoded data to generate 

second coefficient data; 10 
quantizing the second coefficient data by a step size set 

according to said calculated encoding rate; 
variable length encoding said second quantized data to 

generate a second bit stream; and 
recording said second bit stream on said recording 

medium 

7yAn encoding method, comprising the steps of: 
./receiving an input video signal; 

selectively encoding at least a portion of said input video ^ 
signal by intra-frame or predictive encoding to generate 
first encoded data representing intra-frame or predic- 
tive encoded pictures, respectively; 

transform encoding said first encoded data to^generate 
first coefficient data; ' 25 

quantizing said first coefficient data by a fixed step size; 
variable length encoding said first quantized data to 

generate a first bit stream; 
counting a data quantity of said first bit stream every 

predetermined time to indicate a difficulty of encoding; 30 
determining said difficulty of encoding said first bit 

stream based on intra-frame and forward-predictive 

encoded pictures only; 
calculating an allocated code quantity for each unit of 

predetermined time as a function of said difficulty of 35 

encoding so that said allocated code quantity is set to be 

larger for complex pictures and smaller for simple 

pictures; 

selectively encoding said input video signal by intra- 
frame or predictive encoding to generate second 40 
encoded data; 

transform encoding said second encoded data to generate 

second coefficient data; 
quantizing said second coefficient data at a step size set in 

response to said allocated code quantity; 45 
variable length encoding said second quantized data to 

generate a second bit stream; and 
outputting said second bit stream 

8. A method according to claim 7, wherein the fixed 
quantization step size equals one. 50 

9. A method according to claim 7, further comprising the 
step of storing said second bit stream in a buffer before it is 
output, wherein said quantization step size is further based 
on a quantity of said second bit stream stored in said buffer. 

10. A method according to claim 7, wherein said prede- 55 *~ 
termined time is a frame. 

11. An encoding apparatus, comprising: 
/ means for receiving an input video signal; 

means for selectively encoding at least a portion of said 
input video signal by intra-frame or predictive encod- 60 
ing to generate first encoded data representing intra- 
frame or predictive encoded pictures, respectively; 

means for transform encoding said first encoded data to 
generate first coefficient data; 

means for quantizing said first coefficient data by a fixed 65 
step size; 
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means for variable length encoding said first quantized 
data to generate a first bit stream; 

means for counting a data quantity of said first bit stream 
every predetermined time to indicate a difficulty of 
5 encoding; 

means for determining said difficulty of encoding said 
first bit stream based on intra-frame and forward- 
predictive encoded pictures only; 

means for calculating an allocated code quantity for each 
10 unit of predetermined time as a function of said diffi- 
culty of encoding so that said allocated code quantity is 
set to be larger for complicated pictures and smaller for 
simple pictures; 

means for selectively encoding said input video signal by 
15 intra-frame or predictive encoding to generate second 
encoded data; 

means for transform encoding said second encoded data 

to generate second coefficient data; 
means for quantizing said second coefficient data at a step 
20 size set in response to said allocated code quantity; 
means for variable length encoding said second quantized 

data to generate a second bit stream; and 
means for outputting said second bit stream. 

12. An apparatus according to claim 11, wherein the fixed 
25 quantization step size equals one. 

13. An apparatus according to claim 11, further compris- 
ing a buffer for storing said second bit stream before it is 
output, wherein said quantization step size is further based 
on a quantity of said second bit stream stored in said buffer. 

30 14. An apparatus according to claim 11, wherein said 
predetermined time is a frame. 

15s*A recording medium on which there is recorded a 
4S $#cond bit stream obtained by: 
receiving an input video signal; 
35 selectively encoding at least a portion of said input video 
signal by intra-frame or predictive encoding to generate 
first encoded data representing intra-frame or predic- 
tive encoded pictures, respectively; 
transform encoding said first encoded data to generate 
40 first coefficient data; 

quantizing said first coefficient data by a fixed step size; 
variable length encoding said first quantized data to 
generate a fkst bit stream; 
" counting a data quantity of said first bit stream every 
45 predetermined time to indicate a difficulty of encoding; 
determining said difficulty of encoding said first bit 
stream based on intra-frame and forward-predictive 
encoded pictures only; 
calculating an allocated code quantity for each unit of 
50 predetermined time as a function of said difficulty of 
encoding so that said allocated code quantity is set to be 
larger for complicated pictures and smaller for simple 
pictures; 

selectively encoding said input video signal by intra- 
55 frame or predictive encoding to generate second 
encoded data; 

transform encoding said second encoded data to generate 

second coefficient data; 
quantizing said second coefficient data at a step size set in 
60 response to said allocated code quantity; 

variable length encoding said second quantized data to 

generate a second bit stream; and 
recording said second bit stream on said recording 

medium. 

65 
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\f6. (New) An encoding method for encoding source video data, the method 

comprises the steps of: 

encoding said source video data to generate first encoded data; 

detecting a difficulty of the encoding process of source video data based on bit 
amount of said first encoded data; 

deciding an optimum quantization step size which is varied depending on said 
difficulty so that said quantization step size becomes smaller when said source video data is 
more complexity and said quantization step size becomes larger when source video data to be 
encoded is more simple; and 

encoding said source video data by using said optimum quantization step on 
encoding unit basis. 

17. (New) An encoding method for encoding source video data, the method 
comp^isesthe steps of: 

calculating a difficulty of said source video data, wherein said difficulty 
indicates a complexity of a picture within said source video data; 

deciding an optimum quantization step size which is varied depending on said 
difficulty so that said quantization step size becomes smaller when said source video data is 
more complexity and said quantization step size becomes larger when source video data to be 
encoded is more simple; and 

encoding said source video data by using said optimum quantization step. 

(New) An encoding method for encoding source video data, the method 
comprises the steps of: 
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encoding said source video data to generate first encoded data; 

detecting a difficulty of the encoding process of source video data based on 
amount of said first encoded data; 

calculating an allocated code quantity which is varied depending on said 
difficulty so that said allocated code quantity is more increased when said source video data 
is more complexity and said allocated code quantity is more decreased when source video 
data is more simple; and 

encoding said source video data based on said allocated code quantity. 

(New) An encoding method for encoding source video data, the method 
comprises the steps of: 

calculating a difficulty of said source video data, wherein said difficulty 
indicates a complexity of a picture within said source video data; 

calculating an allocated code quantity which is varied depending on said 
difficulty so that said allocated code quantity is more increased when said source video data 
is more complexity, and said allocated code quantity is more decreased when source video 
data is more simple; and 

encoding said source video data based on said allocated code quantity. 

^S). (New) An encoding method for encoding source video data, the method 
comprises the steps of: 

detecting motion vector of a macro block of said source video data; 

encoding said macro block of said source video data by using said detected 
motion vector to generate first encoded data; 
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detecting a difficulty of the encoding process of source video data based on 
amount of said first encoded data; 

deciding an optimum quantization step size which is varied depending on said 
difficulty so that said quantization step size becomes smaller when said source video data is 
more complexity and said quantization step size becomes larger when source video data to be 
encoded is more simple; and 

encoding said macro block of said source video data by using said optimum 
quantization step and said detected motion vector. 

(New) An encoding method for encoding source video data, the method 
comprises the steps of: 

selecting a predictive mode of a macro block of said source video data; 

encoding said macro block of said source video data bv using said selected 
predictive mode to generate first encoded data; 

detecting a difficulty of the encoding process of source video data based on 

Mr 

amount of said first encoded data; 

deciding an optimum quantization step size which is varied depending on said 
difficulty so that said quantization step size becomes smaller when said sour ce video data is 
more complexity and said quantization step size becomes larger when sourc e video data to be 
encoded is more simple; and *~ 

encoding said macro block of said source video data bv using said optimum 
quantization step and said selected predictive mode. 
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7X (New) An encoding apparatus for encoding source video data, the apparatus 
comprising: 

means for detecting motion vector of a macro block of said source video data; 

first encoding means for encoding said macro block of said source video data 
by using said detected motion vector to generate first encoded data; 

means for detecting a difficulty of the encoding process of source video data 
based on amount of said first encoded data; 

means for deciding an optimum quantization step size which is varied 
depending on said difficulty so that said quantization step size becomes smaller when said 
source video data is more complexity and said quantization step size becomes larger when 
source video data to be encoded is more simple; and 

second encoding means for encoding said macro block of said source video 
data by using said optimum quantization step and said detected motion vector. 

J^/ (New) An encoding apparatus for encoding source video data, the apparatus 
comprising: 

means for selecting a predictive mode of a macro block of said source video 

data; 

first encoding means for encoding said macro block of said source video data 
bv using said selected predictive mode to generate first encoded data; 

means for detecting a difficulty of the encoding process of source video data 
based on amount of said first encoded data; 

means for deciding an optimum quantization step size which is varied 
depending on said difficulty so that said quantization step size becomes smaller when said 
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source video data is more complexity and said quantization step size becomes larger when 
source video data to be encoded is more simple: and 

second encoding means for encoding said macro block of said source video 
data by using said optimum quantization step and said selected predictive mode. 

ew) An encoding apparatus for encoding source video data, the apparatus 

comprising: 

first encoding means for encoding said source video data to generate first 

encoded data: 

second encoding means for encoding said source video data based on supplied 
quantization step size to generate second encoded data: 

transmitting buffer for buffering said second encoded data: and 

control means for detecting a difficulty of the encoding process in said first 
encoding means, and for deciding said quantization step size which is varied depending on 
said difficulty so that said quantization step size becomes smaller when said source video 
data is more complexity and said quantization step size becomes larger when source video 
data to be encoded is more simple, and said quantization step size which is varied depending 
on a remaining capacity of said transmitting buffer so as to suppress overflow and underflow 
in said transmitting buffer. 




(New) An encoding apparatus for encoding source video data, -the apparatus 



comprising: 



encoding means for encoding said source video data based on a quantization 



step size to generate encoded stream: 
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transmitting buffer for buffering said encoded stream: and 

control means for calculating a difficulty of said source video data, wherein 
said difficulty indicates a complexity of a picture within said source video data, and for 
deciding said quantization step size which is varied depending on said difficulty so that said 
quantization step size becomes smaller when said source video data is more complexity and 
said quantization step size becomes larger when source video data to be encoded is more 
simple, wherein said quantization step size which is varied depending on a remaining 
capacity of said transmitting buffer so as to suppress overflow and underflow in said 
transmitting buffer. 
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ABSTRACT 



This invention employs a scheme to allow an input video 
signal to undergo encoding, e.g., predictive encoding, DCT 
processing, quantization at fixed quantization step size and 
variable length encoding to generate first encoded data to 
determine (calculate) allocated code quantity every frame or 
every GOP on the basis of data quantity every predetermined 
time, eg., every frame or every GOP of the first encoded 
data and total quantity of usable data to encode the input 
video signal every predetermined time on the basis of the 
allocated code quantity to generate second encoded data. 
Thus, variable rate encoding such that encoding rate changes 
every predetermined rime is realized. As a result, even if 
pictures (frames) of complicated are successive, there is no 
possibility that quantization step size is caused to be large 
with respect to these pictures as in the conventional appa- 
ratus. Thus, uniform high picture quality can be obtained 
through the entirety. Further, since second encoded data 
obtained in a manner as described above has variable rate, 
in the case where such encoded data is recorded onto picture 
recording media, limited memory capacity can be effectively 
used, and recording time of picture recording media can be 
prolonged. In addition, picture data of high picture quality 
uniform over the entirety can be reproduced from the picture 
recording media. 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

Patentee : Tsuyoshi Oda 

Application to 

Reissue Patent No. : 5,703,646 

Issued : December 30, 1997 

For PICTURE ENCODING METHOD, PICTURE 

ENCODING APPARATUS AND PICTURE 
RECORDING MEDIUM 

DECLARATION OF TSUYOSHI ODA 

As a below-named inventor, I hereby declare that: 

1 . My residence, post office address and citizenship are as stated below next 

to my name. 

2. I verily believe myself to be the original, first inventor of the invention 
described and claimed in Letters Patent No. 5,703,646 and in the specification filed herewith for 
which I solicit a patent. 

3. I hereby state that I have reviewed and understand the contents of the 
aforementioned specification, including the claims. 

4. I acknowledge the duty to disclose information which is material to the 
examination of this application in accordance with Title 37, Code of Federal Regulations, 

§ 1.56(a). 

5. I hereby claim foreign priority benefits under Title 35, United States Code 
§ 1 19 of Application Number 5-105943, filed April 9, 1993 and PCT/JP94/00610 filed April 11, 
1994 and state that no other application for patent or inventor's certificate or any PCT 
international application was filed by me on the same subject matter prior to April 9, 1993. 
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6. I do not know and do not believe that the invention was ever known or 
used in the United States of America before my invention thereof. 

7. I verily believe the original Letters Patent to be wholly or partly 
inoperative or invalid by reason of my claiming more or less that I had right to claim in the 
patent by at least failing to claim a method and an apparatus including: 

(A) An encoding method for encoding source video data, the method 
comprises the steps of encoding said source video data to generate first encoded data; detecting a 
difficulty of the encoding process of source video data based on bit amount of said first encoded 
data; deciding an optimum quantization step size which is varied depending on said difficulty so 
that said quantization step size becomes smaller when said source video data is more complexity 
and said quantization step size becomes larger when source video data to be encoded is more 
simple; and encoding said source video data by using said optimum quantization step on 
encoding unit basis. 

(B) An encoding method for encoding source video data, the method 
comprises the steps of calculating a difficulty of said source video data, wherein said difficulty 
indicates a complexity of a picture within said source video data; deciding an optimum 
quantization step size which is varied depending on said difficulty so that said quantization step 
size becomes smaller when said source video data is more complexity and said quantization step 
size becomes larger when source video data to be encoded is more simple; and encoding said 
source video data by using said optimum quantization step. 

(C) An encoding method for encoding source video data, the method 
comprises the steps of encoding said source video data to generate first encoded data; detecting a 
difficulty of the encoding process of source video data based on amount of said first encoded 
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data; calculating an allocated code quantity which is varied depending on said difficulty so that 
said allocated code quantity is more increased when said source video data is more complexity 
and said allocated code quantity is more decreased when source video data is more simple; and 
encoding said source video data based on said allocated code quantity. 

(D) An encoding method for encoding source video data, the method 
comprises the steps of calculating a difficulty of said source video data, wherein said difficulty 
indicates a complexity of a picture within said source video data; calculating an allocated code 
quantity which is varied depending on said difficulty so that said allocated code quantity is more 
increased when said source video data is more complexity and said allocated code quantity is 
more decreased when source video data is more simple; and encoding said source video data 
based on said allocated code quantity. 

(E) An encoding method for encoding source video data, the method 
comprises the steps of detecting motion vector of a macro block of said source video data; 
encoding said macro block of said source video data by using said detected motion vector to 
generate first encoded data; detecting a difficulty of the encoding process of source video data 
based on amount of said first encoded data; deciding an optimum quantization step size which is 
varied depending on said difficulty so that said quantization step size becomes smaller when said 
source video data is more complexity and said quantization step size becomes larger when source 
video data to be encoded is more simple; and encoding said macro block of said source video 
data by using said optimum quantization step and said detected motion vector. 

(F) An encoding method for encoding source video data, the method 
comprises the steps of selecting a predictive mode of a macro block of said source video data; 
encoding said macro block of said source video data by using said selected predictive mode to 
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generate first encoded data; detecting a difficulty of the encoding process of source video data 
based on amount of said first encoded data; deciding an optimum quantization step size which is 
varied depending on said difficulty so that said quantization step size becomes smaller when said 
source video data is more complexity and said quantization step size becomes larger when source 
video data to be encoded is more simple; and encoding said macro block of said source video 
data by using said optimum quantization step and said selected predictive mode. 

(G) An encoding apparatus for encoding source video data, the apparatus 
comprising means for detecting motion vector of a macro block of said source video data; first 
encoding means for encoding said macro block of said source video data by using said detected 
motion vector to generate first encoded data; means for detecting a difficulty of the encoding 
process of source video data based on amount of said first encoded data; means for deciding an 
optimum quantization step size which is varied depending on said difficulty so that said 
quantization step size becomes smaller when said source video data is more complexity and said 
quantization step size becomes larger when source video data to be encoded is more simple; and 
second encoding means for encoding said macro block of said source video data by using said 
optimum quantization step and said detected motion vector. 

(H) An encoding apparatus for encoding source video data, the apparatus 
comprising means for selecting a predictive mode of a macro block of said source video data; 
first encoding means for encoding said macro block of said source video data by using said 
selected predictive mode to generate first encoded data; means for detecting a difficulty of the 
encoding process of source video data based on amount of said first encoded data; means for 
deciding an optimum quantization step size which is varied depending on said difficulty so that 
said quantization step size becomes smaller when said source video data is more complexity and 
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said quantization step size becomes larger when source video data to be encoded is more simple; 
and second encoding means for encoding said macro block of said source video data by using 
said optimum quantization step and said selected predictive mode. 

(I) An encoding apparatus for encoding source video data, the apparatus 
comprising first encoding means for encoding said source video data to generate first encoded 
data; second encoding means for encoding said source video data based on supplied quantization 
step size to generate second encoded data; transmitting buffer for buffering said second encoded 
data; and control means for detecting a difficulty of the encoding process in said first encoding 
means, and for deciding said quantization step size which is varied depending on said difficulty 
so that said quantization step size becomes smaller when said source video data is more 
complexity and said quantization step size becomes larger when source video data to be encoded 
is more simple, and said quantization step size which is varied depending on a remaining 
capacity of said transmitting buffer so as to suppress overflow and underflow in said transmitting 
buffer. 

(J) An encoding apparatus for encoding source video data, the apparatus 
comprising encoding means for encoding said source video data based on a quantization step size 
to generate encoded stream; transmitting buffer for buffering said encoded stream; and control 
means for calculating a difficulty of said source video data, wherein said difficulty indicates a 
complexity of a picture within said source video data, and for deciding said quantization step size 
which is varied depending on said difficulty so that said quantization step size becomes smaller 
when said source video data is more complexity and said quantization step size becomes larger 
when source video data to be encoded is more simple, wherein said quantization step size which 
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is varied depending on a remaining capacity of said transmitting buffer so as to suppress 
overflow and underflow in said transmitting buffer. 

8. I did not discover that the claims of the original patent claimed more or 
less than I had a right to claim until after the original patent was issued. 

9. No claim was previously presented during prosecution of the above 
referenced issued patent that particularly claimed the method and apparatus described in 
paragraph 7 above. 

10. The errors noted above, as well as any other errors to be corrected herein 
arose without any deceptive intention on my part. 

1 1 . New claims 16-25 submitted with this application particularly point out 
the subject matter which I considered my invention and round out the scope of protection to 
which I am entitled. By the omission of such claims the original patent claims less than I had a 
right to claim. 

I hereby appoint William S. Frommer, Registration No. 25,506, and Dennis M. 

Smid, Registration No. 34,930, of Frommer Lawrence & Haug LLP or their duly appointed 

associate, my attorneys, with full power of substitution and revocation, to prosecute this 

application, to make alterations and amendments therein, to file continuation and divisional 

applications thereof, to receive the Patent, and to transact all business in the Patent and 

Trademark Office and in the Courts in connection therewith, and specify that all communications 

about the application are to be directed to the following address: 

William S. Frommer, Esq. 

c/o Frommer Lawrence & Haug LLP 

745 Fifth Avenue 

New York, New York 10151 

Direct all telephone calls to: (212) 588-0800 to the attention of William S. Frommer. Esq. 
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Wherefore I pray that I may be allowed to surrender the Letters Patent No. 
5,703,646 granted December 30, 1997, whereof Sony Corporation, on whose behalf and with 
whose assent this application is made, is the sole owner, by Assignment, and that Letters Patent 
my be reissued to Sony Corporation for the same invention upon the attached specification. 

I, the undersigned applicant, further declare that all statements made herein of my 
own knowledge are true and that all statements made on information and belief are believed to be 
true; and further that these statements are made with the knowledge that willful false statements 
and the like so made are punishable by fine or imprisonment, or both, under Section 1001 of 
Title 18 of the United States Code, and that such willful false statements may jeopardize the 
validity of the application or any patent issuing thereon. 



Tsuyoshi ODA Date 
Residence: Chiba, Japan 
Citizenship: Japan 

Post Office Address of Inventor: Sony Corporation 

Address: 

7-35, Kitashinagawa 6-Chome 
Shinagawa-ku, Tokyo 141, Japan 
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